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ABSTRACT 


nTwo  principal  dimensions  of  system  cost  analysts  arc  addressed.  First, 
the  comparative  analysis  of  alternative  systems  requires  a  method  of  structuring 
and  eynthesizing  cost  estimates  Ln  order  to  reveal  both  the  total  costs  of  achieving 
effectiveness  and  the  significant  differences  to  be  found  among  various  force - 
mixes.  Second,  the  methods  of  estimating  particular  costs  are  evolving  from 
a  history  of  inappropriate  or  inaccessible  data  and  great  uncertainty  concerning 
the  time  and  cost  of  ultimately  achieving  demonstrated  capability.  Several  im¬ 
portant  avenues  for  further  wo*"k  are  discussed. 


This  is  a  pre -publication  copy  of  a  portion  of  the  forthcoming 
book.  DEFENSE  MANAGEMENT,  by  Stephen  Enke,  Ed. ,  to  be 
puDlisted  by  Prentice-Hail,  Inc.  It  Is  not  to  be  copied,  sold, 
or  resold. 
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COSTING  FOR  SYSTEMS  ANALYSIS 


A  systems  analysis,  or  its  quantitative  portion  which  is  often  called  a  cost- 
effectiveness  analysis,  compares  alternative  system  proposals  for  achievement 
of  a  mission  and  attempts  to  discover  which,  on  quantitative  grounds,  is  pre¬ 
ferred.  Less  ambitiously,  it  reveals  those  variations  tliat  are  decidedly  inferior 
and  assists  in  avoiding  gross  inefficiencies.  It  examines  a  part  of  the  defense 
forces.  A  partial  study  cannot,  by  itself,  determine  the  appropriate  level  of 
mission  achievement,  but  by  evaluating  force -mixes  at  each  of  several  plausible 
levels  within  the  mission  it  assists  the  "higher- order"  comparison  among 
levels  of  alternative  programs. 


THE  ROLE  OF  COST 

In  assessing  alternatives,  the  procedure  may  take  either  one  of  two  fun¬ 
damental  forms.  A  desired  level  of  effectiveness  may  be  specified  and  the 
analysis  seeks  the  way  to  achieve  it  most  economically;  or  a  level  of  expendi¬ 
ture  may  be  specified  and  the  analysis  explores  the  effectiveness  provided  by 
system  variations . 

In  most  cases  there  are  several  meaningful  elements  of  achievement  in  the 
objective.  In  a  non- marketable  activity,  such  as  defense,  these  positive  achieve¬ 
ments  within  a  mission  are  not  readily  represented  by  dollar  magnitudes,  or  by 
any  other  single  common  denominator,  for  the  purpose  of  specifying  a  level  of 
an  activity.  For  this  reason,  the  specified  cost  (fixed -budget)  procedure  is 
often  employed.  This  does  not  provide  a  means  of  evaluating  the  relative 
desirability  of  the  several  elements  of  achievement  in  the  mission;  it  merely 
permits  translation  from  the  measurement  of  resources  to  the  measurement  of 
effectiveness  elements,  for  each  system  examined,  in  order  to  advance  sensible 
comparisons  and  selections. 

Why  should  the  compared  force -mixes  be  structured  deliberately  to  be 
equal  in  cost  (or  effectiveness)?  The  answer  is  straightforward,  -  if  system 
or  force  I  is  demonstrably  more  effective  than  II  and  also  costs  more  than  II, 
we  cannot  determine  from  this  information  whether  I's  greater  effectiveness 
results  simply  from  a  larger  scale  of  expenditure,  or  from  a  more  productive 
mixture  of  resources  and  operational  methods,  or  both.  However,  if  their  cost 
is  equal,  the  more  effective  alternative  is  clearly  preferred.  In  terms  of  eco¬ 
nomic  analysis,  the  study  at  this  stage  attempts  to  locate  a  point  on  the  total 
cost  function,  -  with  cost  as  a  function  of  output  (effectiveness). 

It  is  often  appropriate  to  define  a  bench-mark  force  for  the  postulated 
budget.  In  a  study  to  assist  future  procurements  for  a  current  mission,  one 
bench-mark  may  be  taken  as  a  current  financial  program  with  systematic  ex¬ 
trapolation.  Variations  may  then  be  generated  by  diverting  expenditures  on 
one  subsystem,  say  attack  aircraft,  and  applying  the  amount  to  other  subsystems, 
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say  interceptor  and  reconnaissance  aircraft.  If  the  quantitative  change  in  ef¬ 
fectiveness  associated  with  eacn  decrement  or  increment  is  computed  separately, 
we  have  estimated  portions  of  the  familiar  marginal  productivity  functions  of 
economic  analysis  (reference  (a)). 

The  construction  and  analysis  of  forces  in  this  fashion  makes  it  clear  that 
in  a  comparative  evaluatio  J  srch  proposals  it  is  their  differences  in  resource 
drains  that  count.  Errors  and  omissions  in  estimating  costs  that  are  invariant 
among  structural  variations  do  not  bias  a  comparison.  But  these  costs  must  be 
estimated  if  the  result  is  to  contribute  to  the  "higher-order"  allocation  between 
force  levels  for  different  missions  in  the  military  sector. 

This  is  why  a  partial  or  comparative  systems  analysis  usually  demands 
substantially  more  than  the  bare -minimum  cost  analysis  for  the  comparison 
alone.  Program  decisions  and  budgetary  implementation  within  the  Department 
of  Defense  require  exposure  cf  the  full  incremental  cost  of  a  candidate  system 
and,  ultimately,  programmed  over  time.  Further,  the  wider  the  scope  of  al¬ 
ternatives  evaluated  in  a  particular  systems  analysis  (e.g.,  Navy  vs  Air  Force 
systems)  the  larger  will  be  the  portion  of  costs  that  differ  among  systems,  and 
the  more  extensive  is  the  minimal  cost  analysis  directly  required  for  the  par¬ 
tial  analysis. 

A  selection  of  a  system  affects  the  flow  of  only  a  portion  of  defense  re¬ 
sources.  The  cost  estimates  for  a  systems  analysis  should  identify  incremental 
resource  costs  for  each  system  proposal.  A  proposal  should  be  charged  with 
all  the  costs  which  are  required  for  achieving  the  measured  mission  effective¬ 
ness,  but  exempted  from  charges  that  attach  to  effectiveness  in  an  unevaluated 
mission  or  in  another  era.  By  this  principle,  an  effort  to  attach  costs  to  ef¬ 
fectiveness  can  succeed  in  practice  only  if  the  cost  analysis  is  intimately 
governed  by  the  type  and  time -profile  of  effectiveness  that  is  evaluated.  This 
is  now  conventional  wisdom  in  defense  analysis.  The  difficulty  lies  in  measur¬ 
ing  additional  resource  claims. 

Cost  measurements  employ  monetary  prices  and  for  most  issues  there  is 
no  better  way.  An  exception  occurs  in  tightly-constrained  operational  analyses 
of  tasks  in  which  the  physical  quantities  of  certain  resources  are  absolutely 
fixed.  If  certain  resources  are  to  be  employed  only  in  the  analyzed  task  with 
no  additional  quantities  available  from  other  uses,  monetary  prices  observed 
in  exchange  transactions  elsewhere  are  irrelevant.  Without  substitutability  of 
resources  among  uses  the  suboptimization  is  a  purely  internal  maximization, 
subject  to  the  requirement  that  it  is  consistent  with  broader  defense  objectives. 

With  this  exception  (ignoring  imputed  costs  for  the  moment),  system  and 
cost  analysts  in  practice  employ  actual  money  prices  observed  in  exchange 
transactions.  Resources  claimed  by  a  selected  project  must  be  paid  dollar 
prices  sufficient  to  attract  and  hold  them  to  defense  from  their  alternative  op¬ 
portunities.  Prices  thus  express  fundamental  economic  costs  of  defense  in  our 
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economy,  as  well  as  constraints  on  a  decision-maker's  ability  to  achieve  ob¬ 
jectives.  (At  the  end  of  this  chapter  the  situation  is  cited  in  which  observed 
prices  are  imperfect  reflectors  of  true,  alternative  costs.) 

The  cost  analysis  of  systems  studies  may  be  given  two  dimensions:  (1) 
Synthesis  and  summary  of  full  system  costs  for  the  comparison  of  alternative 
force-mixes.  (2)  Research  on  component,  subsystem,  and  associated  costs,  - 
work  that  involves  specific  development  and  application  of  cost  estimating  data 
and  techniques . 

In  a  systems  study  analysts  find  themselves  simultaneously  resolving  issues 
in  both  dimensions.  For  each  force  variant  the  analysis  estin.ates  the  resource 
combinations  and  associated  cost  of  the  structure  providing  the  capability.  The 
cost  analyst  should  work  with,  rather  than  simply  transmit  exhibits  to  analysts 
who  structure  and  compare  the  effectiveness  of  candidate  alternatives  so  that 
the  cost  of  each  system  is  properly  estimated  and  summarized  in  relation  to  the 
effectiveness  that  results. 


SYNTHESIS  OF  SYSTEM  COSTS 

There  are  many  ways  of  splitting  and  then  summarizing  a  total  system  cost. 
The  process  ought  to  be  inspired  by  a  simple  theoretical  guideline.  If  we  had 
the  power  to  maM  good  estimates  of  all  the  necessary  variables  and  parameters, 
and  the  time  to  do  so,  we  would  determine  for  each  course  of  action  the  time- 
pattern  of  mission  effectiveness  and  the  time -pattern  of  associated  costs  that 
achieve  it.  Further,  if  we  could  demonstrate  the  appropriateness  of  some  rate 
of  discount  with  which  to  reduce  magnitudes  in  different  years  to  equivalence, 
we  could  discount  future  terms  to  calculate  present  values  of  cost.  Then  we 
could  seek  the  preferred  effectiveness  for  a  given  present  value  of  cost,  or  the 
minimum  present  value  of  cost  for  any  given  stream  of  effectiveness . 

In  practice,  the  application  of  this  model  is  not  easy.  Various  expedients 
and  rules  of  thumb  are  used.  The  authors  have  at  some  time  used  many  of 
these  and  been  tempted  by  most.  We  will  highlight  some  problems  and  prac¬ 
tices  in  cost  analysis  with  an  example  that  introduces  a  discussion  of  synthesis 
and  estimation. 


An  Example  of  Force  Costs 

Both  ground  systems  and  air  systems  contribute  to  the  same  miss  on  cap¬ 
ability.  Currently  there  are  40  ground  units  and  12  air  units  in  the  force  struc¬ 
ture.  Expansion  of  the  capability  to  a  higher  level  is  considered,  including 
either  improved  ground  or  air  units,  or  both.  For  simplicity  we  construct 
three  alternative  incremental  mixes  only.  The  example  illustrates  the  problem 
of  comparing  costs  if  the  forces  are  estimated  to  have  equal  effectiveness,  and 
suggests  the  necessity  of  reconstructing  the  illustrated  incremental  forces  if 
they  are  to  bt  made  equai-cost  in  some  acceptable  sense. 


Each  mix  is  an  increment  to  the  current  force  of  40  ground  and  12  air  units. 
Mix  I  is  a  pure  ground  increment,  mix  II  includes  both  ground  and  air  units,  and 
mix  III  is  a  pure  air  increment.  They  are  as  follows: 

L  II  HI 

Ground  Units  60  (+20)  50  (+10)  40  (+0) 

Air  Units  12  (+  0)  14  (+  2)  17  (+5) 

The  "effective”  life  is  estimated  to  be  15  years  for  new  air  units  and  five 
years  for  new  ground  units.  Non-recurring  costs  per  unit  of  air  and  ground  are 
estimated  as  functions  of  the  volume  produced.  As  shown  in  figure  1,  per-unit 
initial  investment  costs  decline  vtien  production  efficiencies  are  achieved  at  in¬ 
creased  volume.  The  recurring  cost  is  estimated  to  be  a  constant  $50/ unit  'year 
for  air  and  $10/unit/year  for  ground. 


Ground 

Units 


FIG.  1:  CUMULATIVE  AVERAGE  COSTS  OF  NEW  UNITS 
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Expenditures  are  then  summed  to  get  a  total  cost  for  each  force-mix  shown 
in  table  1. 


TABLE  I 
TOTAL  COSTS 


If  effectiveness  is  desired  over  five  years  only  and  total  costs  are  compared 
only  for  that  interval,  the  pure  ground  system  has  the  lowest  cost.  If  the  three 
forces  promise  equal  effectiveness  over  five  years  the  air  units  cannot  compete 
because  of  their  high  non-recurring  costs. 

Suppose  effectiveness  is  wanted  beyond  five  years  but  the  cost  comparison 
is  limited  to  the  short  interval.  If  air  units  are  selected  now  rather  than  ground 
units,  at  the  end  of  five  years  such  air  units  would  then  provide  a  remaining  ef¬ 
fectiveness  that  permits  the  avoidance  of  later  expenditures  that  otherwise  would 
be  necessary  for  replacements  of  any  ground  units  if  selected  now.  A  side  cal¬ 
culation  nr  ay  take  into  account  this  end-of-study  "remaining  value"  of  durable 
systems. 

It  can  be  shown  that  the  remaining  value  of  air  units  actually  in  the  inventory 
five  years  hence,  is  the  excess  of  subsequent  expenditures  (beyond  five  years) 
that  would  have  been  incurred  on  ground  units  (if  selected  initially)  above  sub¬ 
sequent  expenditures  on  air  units  that  were  selected  initially,  given  that  either 
selection  can  provide  a  specified  effectiveness.  This  excess  is  the  net  expendi¬ 
ture  that  can  be  avoided  in  the  period  after  five  years  if  air  units,  rather  than 
ground  units  are  selected  initially  to  provide  effectiveness  over  the  entire  longer 
interval. 

If  remaining  value  is  properly  accounted  for.  the  time  horizon  must  be  ex¬ 
tended  as  a  side  calculation  and  the  remaining  value  then  quoted  separately, 
accompanying  the  five-year  total  costs.  It  is  simpler  to  extend  the  time  horizon 
for  the  main  study  to  include  future  procurement  and  operation  of  ground  units 
as  replacements.  As  the  study  interval  lengthens  the  remaining  value  of  air 
units  is  deferred  and  reduced.  It  vanishes  if  an  interval  is  constructed  so  that 
across  all  three  structures  the  end  of  every  chain -of -assets  coincides.  This 
interval  in  the  illustration  is  15  years.  For  this  horizon,  total  costs  of  II  and  III 
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are  about  equal  and  below  the  total  cost  of  the  pure  ground  increment.  The 
relationship  between  total  cost  and  time  is  shown  in  figure  2. 

Clearly  the  greater  early  costs  in  forces  II  and  III  above  force  I  are  more 
than  counterbalanced  by  lower  costs  in  the  future.  These  are  undiscounted  costs. 
Perhaps  the  discounted  15  year  costs  are  desired.  The  present  values  of  I  and 
III  are  equal  at  a  discount  rate  of  approximately  8  percent.  For  I  and  Ii.  the 
rate  for  equating  present  value  is  higher,  approximately  13  percent,  because  the 
penalty  in  early  expenditure  of  II  is  much  smaller  than  of  III. 

Quantitative  differences  in  total  cost  are  dependent  upon  the  estimates  in  the 
various  cost  categories.  If  the  recurring  cost  of  the  air  system  is  smaller, 
$35/unit/year,  cumulative  force  costs  are  shown  in  figure  3.  Force-mix  I  stili 
has  a  significant  b'it  narrower  cost  advantage  at  five  years.  Differences  at  15 
years  are  now  much  larger.  A  discount  rate  of  20  percent  is  now  required  to 
reduce  the  15  year  cost  of  I  to  equality  with  that  of  III,  and  25  percent  makes  the 
present  value  of  !  and  II  equal. 


Cost  Categories 

A  useful  step  in  cost  analysis  is  the  preparation  of  appropriate  categories 
of  relevant  costs.  Cost  categories  assist  in  separating  expenditures  into  quasi- 
homogeneous  types  that  are  distinguished  by  particular  resources,  activities, 
and  by  the  causes  that  determine  their  amount.  Usually  costs  will  divide  ac¬ 
cording  to  whether  they  are  non-recurring  or  recurring,  and  into  majoi  categories 
such  as  R&D.  investment,  and  operations  and  maintenance  (reference  (b)). 

Elements  similar  to  those  listed  in  table  I  are  usually  included  in  a  weapon 
system  cost  analysis.  The  three  major  categories  follow  a  chronological  order: 
however,  there  is  usually  some  time  overlap.  Investment  expenditures  occ>:4 
before  the  completion  of  research  and  development,  and  operations  expenditures 
begin  before  the  delivery  of  all  items  of  prime  mission  equipment. 

There  are  situations  where  a  categorization  exactly  as  shown  in  table  II  is 
misleading.  An  example  is  a  study  of  alternative  means  of  placing  payloads  into 
orbit  around  the  earth.  In  most  military  systems  the  cost  of  launching  rockets 
would  be  considered  an  initial  investment.  However,  for  keeping  earth  satel¬ 
lites  continuously  in  orbit  boosters  would  be  launched  periodically.  If  they  arc 
non -recoverable,  their  costs  should  be  treated  ',r  operations  costs  similar  to 
other  recurring  costs. 
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TABLE  II 


TYPICAL  CLASSIFICATION  OF  WEAPON  SYSTEM  COST  ELEMENTS 


A.  Non-Recurring 

i.  Research  and  Development 

1 .  Preliminary  Research  and  Design  Studies 

2.  Design  and  Development  (Of  Subsystems) 

3.  System  Test  (Of  Complete  System) 

II.  Initial  Investment 

1.  Prime  Mission  Equipment 

2.  Support  Equipment 

3.  Initial  Spares,  Spare  Parts  and  Stocks 

4.  Initial  Training 

5.  Initial  Travel,  Transportation,  and  Miscellaneous 

6.  Military  Installations 

B.  Recurring 

III.  Annual  Operations 

1 .  Pay  and  Allowance 

2.  Equipment  and  Installations  Replacement 

3.  Equipment  and  Installations  Maintenance 

4.  Replacement  Training 

5.  Consumables  (e.g.,  fuel,  oil,  etc.) 

6.  Recurring  Travel,  Transportation,  and  Miscellaneous 


The  cost  of  altering  ships  and  their  weapons  and  electronic  systems  for  the 
purpose  of  improving  capability  presents  a  problem  of  categorization.  During 
her  lifetime,  a  ship  may  be  altered  several  times  but  at  irregular  intervals 
because  major  alternations  are  not  physically  necessary  to  achieve  the  major 
capability  of  the  ship.  Cost  models  which  include  an  estimate  for  alterations 
averaged  as  an  annual  expense  will  be  inappropriate  where  the  cost  of  operating 
ships  for  their  full  life-times  is  not  at  issue.  Cost  categories  should  depend 
upon  the  application  and  each  analysis  should  assess  its  special  needs. 
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Time-Patterning  the  Estimates 

Another  virtue  of  cost  categories  is  that  they  assist  an  analyst  in  focusing 
his  estimates  over  a  series  of  future  years.  It  is  useful  to  disaggregate  con¬ 
siderably  in  order  to  make  credible  estimates  of  the  annual  pattern  of  costs. 
When  similar  costs  are  more  remote  and  uncertain,  in  the  case  of  advanced 
proposals,  this  detail  is  less  useful. 

Estimates  of  cost  time-streams  serve  several  purposes:  (1)  Some  elements 
of  cost  are  typically  deferred  and  sometimes  omitted  from  analysis  unless  the 
procedure  calls  specifically  for  time -patterning  costs;  examples  are  infrequent 
ship  modernizations  or  aircraft  model  improvements.  (2)  Decision-makers 
must  anticipate  when  and  what  types  of  expenditures  will  occur,  because  the 
budgetary  process  authorizes  expenditures  annually  and  by  specific  types.  (3) 
Decision-makers  are  not  indifferent  between  expenditures  in  different  years 
even  if  the  arithmetic  total  over  a  period  is  given.  Some  of  the  most  important 
substitution  possibilities  within  defense  are  substitutions  over  time,  -  e.g. , 
maintaining  existing  forces  longer  and  developing  replacements  later,  or  vice 
versa. 


Time  Horizon  and  Remaining  Value 

The  study  interval  is  appropriately  determined  within  the  individual  analysis. 
No  general  convention  or  rule  of  thumb,  apart  from  the  purpose  of  the  analysis, 
can  specify  it  except  arbitrarily.  Within  the  analysis,  cost  and  production  feas¬ 
ibility  considerations  alone  cannot  specify  it.  Analysts  should  confront  the 
questions  of  what  the  characteristic  threat  is,  what  types  of  capability  may  met 
it,  how  long  each  operational  system  may  be  expected  to  provide  effectiveness, 
what  the  prospective  future  equipments  and  methods  may  be,  and  when  develop¬ 
ment  may  provide  them . 

As  illustrated  earlier  in  the  example,  some  systems  may  promise  further 
effectiveness  beyond  a  chosen  time  horizon.  This  may  be  credited  to  the  air 
units  at  a  five-year  horizon  by  estimating  expenditures  in  the  still-farther  future 
that  may  be  avoided  if  those  units  are  then  in  the  inventory.  If  assets  are  not 
expected  to  be  very  effective  in  the  primary  mission  there  may  be  a  secondary 
role  where  their  availability  will  permit  certain  expenditures  to  be  avoided. 

Even  when  longer  time-horizons,  such  as  15  years,  are  employed  we  may 
expect  assets  of  some  forces  to  be  effective  beyond  the  study  interval.  We 
usually  lack  reliable  evidence  or  techniques  for  estimating  remote  effectiveness 
and  the  remote  avoidable  expenditure  the  assets  permit.  Cost  analyses  some¬ 
times  employ  the  arithmetic  of  amortization  as  an  implicit  predictor.  Care 
must  always  be  taken  to  avoid  substituting  simple  arithmetic,  because  it  is 
simple,  for  attempts  at  estimation  or  analysis .  However,  even  an  arbitrary 
accounting  method  will  frequently  give  a  far  better  estimate  of  remaining  value 
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at  the  end  of  a  study  interval  than  if  the  question  is  ducked  and  the  value  set  at 
zero.  One  can  avoid  the  question  but  he  cannot  thereby  avoid  making  an  implicit 
estimate  -  and  it  may  be  a  bad  one . 

A  related  issue  involves  existing  assets  proposed  for  use  in  a  system  under 
evaluation.  Is  such  equipment  to  be  treated  as  free  when  "inherited'’  by  a  system, 
or  not?  Almost  any  comparison  of  alternative  systems  involves  existing  assets, 
although  many  will  be  specific  to  a  particular  purpose  with  no  meaningful  alter¬ 
native  use.  Nevertheless,  rather  than  assume  this,  the  analysis  should  ask  if 
the  asset  is  useful  for  another  objective  and  what  the  maximum  expenditure 
which  could  be  avoided  by  its  application  elsewhere  may  be.  This  is  the  (alter¬ 
native)  cost  of  "assigning"  it  to  a  system  in  the  analysis.  If  this  factor  becomes 
important  the  study  may  consider  a  widening  of  scope  in  an  attempt  to  include 
more  types  of  effectiveness. 

Current  alternative  values  and  future  remaining  values  are  imputations . 
Estimates  of  net  costs  involving  imputations  appear  less  straightforward,  and 
therefore  less  reliable,  than  estimates  of  most  explicit  costs.  To  be  convincing, 
an  estimate  of  remaining  value  must  demonstrate  that  the  asset  structure  pro¬ 
mises  a  desired  effectiveness  in  some  mission  in  the  farther  future.  It  is  little 
wonder  that  cost  analysts  are  unenthusiastic  about  imputations  that  rest  on  ef¬ 
fectiveness  measures.  Yet  these  imputations  do  not  introduce,  but  merely 
underscore,  an  analytic  difficulty  of  effectiveness  measures. 


Time  Preference 

Economists  differ  in  their  interpretation  of  what  economic  analysis  implies 
about  applying  a  discount  rate  in  comparing  non- marketable  projects  such  as 
defense.  One  opinion  says  to  discount  at  the  rate  of  interest  on  "comparable" 
types  of  investment  in  the  market  economy.  The  rationale  is  that  when  we  ex¬ 
tract  resources  to  the  non- marketable  sector  from  other  uses  we  cannot  be  ef¬ 
ficient  unless  we  select  only  from  the  systems  that  are  as  productive  of  future 
benefits,  or  as  saving  in  future  expenditures,  as  successful  investments  in  the 
private  sector. 

Another  opinion  says  that  economic  analysis  cannot  be  extended  to  demon¬ 
strate  the  achievement  of  efficiency  from  testing  non -marketable  investments 
with  the  market  rate  of  interest  appropriate  for  testing  productive  investments 
by  individuals  who  compete  for  resources  under  private  property.  Although  this 
opinion  can  be  quite  unsparing  in  criticizing  unevaluated  non -marketable  projects, 
it  rejects  the  direct  transferability  of  a  market  rate  of  interest  to  assist  evalu¬ 
ation  of  these  activities  (reference  (c)). 

Another  question  concerns  discounting  cost  streams  "to  take  account  of 
risk".  Loading  the  discount  rate  with  a  factor  "for  risk"  is  quite  perverse  if  it 
is  intended  to  correct  for  doubts  about  the  relative  cost  estimates  in  the  systems 
studied. 
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This  is  shown  by  considering  two  estimated  cost  streams  for  producing  a 
given  effectiveness.  Suppose  we  know  from  empirical  evidence  that  cost  esti¬ 
mates  for  the  farther-future  become  less  reliable.  Applying  a  discount  rate  to 
streams  of  single -valued,  annual  cost  estimates  reduces  remote  costs  more 
than  early  costs.  The  discounting  operation  thereby  biases  choice  toward  pro¬ 
jects  with  relatively  large  later  cost  estimates,  the  more  uncertain  alternatives. 
The  bias  would  be  still  stronger  if  an  attempt  is  made  to  account  for  differential 
estimating  reliability  among  entire  streams  by  discounting  riskier  cost  streams 
at  higher  rates. 

Adding  a  risk  factor  to  a  discount  rate  is  sometimes  suggested  for  a  different 
reason:  our  inability  to  anticipate  options  we  later  find  attractive.  There  is 
much  experience  in  which  options  were  initially  unanticipated,  because  of  either 
ignorance  or  myopia  concerning  the  real  nature  of  the  threat  or  technology,  and 
later  discovered  to  be  attractive  as  information  grew.  Future  evidence,  if  not 
too  late  to  impact  a  selection,  will  cause  us  to  reevaluate  an  initial  selection  and 
perhaps  to  abandon  it.  Will  loading  comparisons  with  a  discount  rate  aid  such 
sequential  decision-making?  In  a  sense,  yes,  because  it  shifts  choice  away 
from  projects  with  relatively  heavy  early  costs,  thereby  dampening  early  com¬ 
mitments.  It  is,  however,  hard  to  imagine  anything  more  arbitrary  than  loading 
a  discount  rate  as  a  device  to  promote  incremental  spending  in  search  of  reliable 
information  on  obscure  benefit  and  costs. 


Cost -Summarizing  Techniques 

A  number  of  conventions  for  summarizing  costs  may  be  found.  Several  that 
have  been  employed  in  studies  are  defined  here  (reference  (d)). 

Five-Year  System  Costs:  This  convention  arithmetically  sums  the  R&D, 
initial  investment,  and  five  times  th~  annual  cost  of  operating  the  system  at  a 
specified  level.  Remaining  values  at  the  end  of  five  years  of  level  operations 
(which  may  be  7-15  years  after  the  initiation  of  R&D)  are  excluded.  Also  ex¬ 
cluded  are  'build-up  costs",  i.e.,  the  operating  costs  incurred  during  the  phase- 
in  period  before  the  system  reaches  its  full  force. 

Period  Outlay;  Build-up  costs  are  included  and  outlays  are  time-phased, 
either  by  year  of  obligation  or  expenditure.  In  some  studies,  remaining  value 
has  been  referenced  simply  by  listing  the  age  structure  of  assets  as  of  the  study’s 
cut-off  date. 

Net  Cost:  This  technique  attempts  a  measurement  of  remaining  values. 
Specifically,  it  gives  effect  to  unequal  useful  lifetimes  among  the  principal  assets 
both  within  and  acrot  s  force-mixes.  The  time  pattern  of  costs  is  estimated, 
including  a  remaining  value  of  assets  at  the  cut-off  date.  This  is  then  subtracted 
from  the  estimated  expenditures  to  get  a  "net  cost" . 
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Equal  Life- Lengths  Cost:  An  approximate  time  period  is  computed  so  that 
across  alt  force-mixes  the  end  of  each  estimated  chain-of-assets  roughly  coin¬ 
cides.  A  special  case  occurs  when  in  each  chain  the  replacement  equipments 
are  expected  to  have  the  same  useful  lifetime  as  initial  equipments  in  the  chain. 
This  is  illustrated  by  the  five-year  replications  of  ground  units  in  the  example 
above . 

There  has  been  over  emphasis  of  the  five-year  system  cost  technique  in  the 
past.  Because  of  its  exclusions,  it  can  be  misleading  in  some  cases.  But  this 
is  not  meant  to  suggest  that  the  five-year  cost  has  no  place  in  cost  analysis. 

The  five-year  system  cost  estimate  is  useful  in  a  pre -development  analysis  of 
systems  that  rely  upon  technological  advances  far  in  the  future.  An  aerospace 
plane  or  a  nuclear  powered  missile  impose  requirements  and  associated  costs 
that  cannot  now  be  estimated  with  accuracy  sufficient  to  warrant  the  use  of  de¬ 
tailed  cost  summaries. 

For  less  advanced  systems  nearing  procurement,  whose  specifications  can 
be  estimated  with  more  accuracy,  one  should  not  ignore  the  time  pattern  of 
costs,  unequal  lifetimes  of  alternatives,  build-up  costs,  etc.  Feasible  produc¬ 
tion,  construction,  and  introduction  schedules  should  be  examined  more  care¬ 
fully.  A  satisfactory  resolution  should  be  found  for  the  question,  "Are  we 
examining  optional  mixes  having  a  useful  capability  over  the  same  time  period?" 
At  this  stage,  reliance  on  a  short  study  interval  and  effectiveness  measurements 
for  only  one  nominal  future  date  can  obscure  real  issues  involved  in  selection  of 
a  production  program. 


TOPICS  IN  COST  ESTIMATION 

Broadly,  there  are  two  types  of  decisions  in  which  systems  analyses  assist: 
force-structure  procurement  and  development  planning.  The  following  topics 
relate  to  both. 


Use  of  Historical  Data 

Although  sunk  costs  are  irrelevant  to  evaluations  of  future  options,  records 
of  past  costs  are  indispensable  as  a  base  upon  which  estimates  of  future  costs 
and  system  characteristics  are  projected.  Prospective  systems  usually  involve 
components  and  operations  that  have  close  analogs  in  the  past  or  present.  Past 
or  current  experience  also  warns  us  of  pitfalls,  delays,  and  improvisations 
having  substantial  cost  consequences. 

It  takes  years  to  build  a  good  library  of  data  on  both  procurement  and  oper¬ 
ational  aspects  of  defense  programs.  The  task  is  riddled  with  frustrations. 
Government  record-keeping  has  always  been  adapted  primarily  to  assuring  fidel¬ 
ity  to  financial  authorizations.  These  records  are  not  ordinarily  useful  in  relating 
means  to  ends  -  in  evaluating  activities  by  their  claims  on  resources  and  their 
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contributions  to  some  agreed  objective  (reference  (e)).  Enormous  efforts  have 
been,  and  are  still,  required  to  adapt  them  to  this  function. 

Defense -related  cost  data  are  often  treated  as  sensitive  information  by 
government  agencies  and  as  privileged  information  by  industrial  contractors . 

(This  is  unrelated  to  military  security. )  One  effect  is  to  reduce  the  information 
available  to  cost  analysts  outside  the  government  proper.  Although  certain  in¬ 
formation  is  rightly  privileged  to  private  enterprise,  impediments  here  are 
gradually  eroding.  The  military  services  have  instituted  publication  of  general 
program -planning  and  cost  factors.  Contractors  are  required  to  submit  certain 
cost  reports  depending  upon  the  type  of  contract.  These  are  very  useful  sources 
of  data  but  require  much  evaluation.  There  is  often  inconsistency  across  sources. 
An  investigator  who  wants  detail  usually  must  probe  behind  the  reports  for  it. 


Cost -Quantity  Relationships 

Hardware  procurement  costs  have  received  mere  attention  than  operating 
costs.  The  chief  relationship  established  lor  major  items  of  military  hardware 
is  the  dependency  of  total  production  cost  per  unit  upon  volume  of  output.  Cost 
per  unit  is  observed  to  decline  with  increased  production  over  an  extended  range 
of  output.  This  has  been  observed  in  the  airframe  industry  and  it  has  found  ap¬ 
plication  elsewhere  among  major  equipments  produced  in  quantity  (reference  (f)). 
Marked  reductions  in  man-hours  used  in  production  of  successive  lots  are  ob¬ 
served.  One  explanation  is  "learning".  In  the  course  of  operations,  workers 
and  supervisors  learn  the  process  and  how  to  do  it  better.  Data  on  other  inputs 
also  indicate  a  similar  effect  for  materials .  Cost  analysts  have  by  now  fitted  a 
number  of  functions  relating  cumulative  cost  to  cumulative  quantity. 

Another  influence  of  total  volume  on  cost  is  difficult  in  practice  to  separeate 
from  the  first.  In  planning  production  there  is  an  initial  choice  of  how  much 
investment  in  plant,  equipment,  and  special  tooling  to  undertake.  For  a  planned 
cumulative  volume,  trade-offs  are  possible  between  initial  investment  costs  and 
subsequent  operating  costs.  As  the  intended  volume  (for  which  initial  plans  are 
made)  increases  over  a  significant  range,  a  firm  can  plan  the  production  method 
to  select  more  diverse  and  durable  equipment  and  tooling,  thereby  lowering  per- 
unit  costs  of  production  (reference  (g)').  This  proposition  has  received  less  em¬ 
pirical  testing  than  "learning",  but  it  is  familiar  to  defense  contractors  who 
produce  specific  equipments  to  order.  It  may  deserve  more  investigation  in  plan¬ 
ning  the  volume  production  of  items  such  as  ships  or  the  huge  new  CSA  transport 
aircraft.  Cost  analysis  in  this  context  strives  to  reveal  the  efficient  (economical) 
technology  to  employ  In  major  defense  production. 
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Estimating  Procedures 

For  an  operational  (or  retired)  aircraft  an  analyst  may  find  production 
costs  on,  say,  four  to  eight  delivered  lots  produced  by  a  specific  firm.  In 
estimating  future  costs  of  a  proposed  aircraft  with  different  attributes,  this  in¬ 
formation  represents  one  observation  only.  The  objective  may  be  to  estimate 
the  future  research  and  development,  procurement,  and  operating  costs  of  a 
proposed  aircraft  that  flies  faster,  lands  slower,  carries  more  payload,  and  has 
an  all-weather  capability.  Such  an  aircraft  will  attempt  aerodynamic  advances. 

It  will  weigh  more,  have  bigger  and  perhaps  more  efficient  engines,  use  different 
alloys,  mount  advanced  sensors  and  weapons.  Its  configuration  may  imply  dif¬ 
ferent  maintenance  practices.  Perhaps  its  operational  mode  implies  different 
attrition.  The  cost  analyst  enumerates  the  delivered  aircraft  that  differ  in  these 
distinctive  factors  but  are  sufficiently  homogeneous  in  other  respects  so  that 
their  recorded  costs  may  bear  directly  on  estimation  for  the  new  vehicle.  If  he 
finds  four  to  eight  observations  on  other  "similar  but  different”  vehicles  he  feels 
an  embarrassment  of  riches  in  a  predicament  that  apalls  most  statisticians . 

In  fact,  there  is  perhaps  now  more  good  information  on  aircraft  and  their 
components  than  on  any  other  type  of  major  vehicle  or  equipment.  There  is  now 
enough  data  in  some  areas  to  warrant  use  of  multiple  regressions.  Estimated 
regression  relationships  for  airframes,  engines,  ships,  and  certain  aspects  of 
aircraft  and  ship  maintenance  have  now  appeared  (reference  (h)). 

These  must  be  tested  by  their  fruitfulness  -  do  they  assist  us  in  better  cost 
estimating  than  we  perform  without  them?  Few,  if  any,  of  the  "samples"  can  be 
said  to  exhibit  the  statistical  properties  for  which  regression  is  derived  in  sta¬ 
tistical  theory.  The  application  of  statistical  estimating  relationships  to  an 
estimating  problem  must  be  made  with  caution,  particularly  when  forecasting 
beyond  the  limits  of  the  data.  An  experienced  researcher  will  also  employ  judg¬ 
ment  based  on  information  "outside"  the  regression  if  the  subject  forecast  is  for 
an  item  that  differs  in  important  attributes  from  the  events  used  in  estimating  the 
regression. 

Most  cost  estimates  must  be  accomplished  with  too  few  observations  on 
particular  subsystems  or  components  to  permit  regression  estimation  of  relation¬ 
ships.  When  data  are  sparse  the  estimator  may  use  two  types  of  procedures. 

First,  he  may  look  for  one  or  more  close  analogs  for  which  information  is 
available.  For  example,  in  estimating  the  cost  of  anew  re-entry  subsystem  he 
will  examine  in  some  detail  the  cost  of  the  one  or  two  recent  vehicles  that  re¬ 
semble  the  candidate  design.  In  such  procedures  it  is  common  to  try  to  account 
for  systematic  effects  of  different  components  on  the  total  cost  of  the  vehicle, 
and  to  treat  cost  elements  such  as  labor,  material,  and  engineering-hours 
separately.  Experience  has  shown,  however,  that  detailed  estimates  can  still 
be  bad  estimates.  For  advanced  proposals,  this  is  more  likely  if  an  analyst 
attempts  to  relate  estimates  exclusively  to  forecasted  component  weights  rather 
than  to  prescribed  performance.  Detail  frequently  implies  credibility  to  the 
uninformed.  Perhaps  this  accounts  to  some  extent  for  the  detail  one  often  finds 
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in  cost  estimates.  If  the  estimation  exhibits  are  very  detailed  it  can  be  difficult 
to  distinguish  a  paucity  from  a  plenty  of  reliable  and  independent  observations. 

By  contrast  with  the  specific  analog,  a  second  approach  when  independent 
observations  are  sparse  is  to  use  cost  information  that  is  aggregated  to  a  higher 
level.  Where  information  has  not  been  disaggregated  to  relate  it  to  characteris¬ 
tics  of  individual  units,  it  is  often  available  from  budgetary  exhibits  on  an  over¬ 
all  basis.  This  type  of  information  has  been  employed  in  estimating  costs  of 
operation  and  support.  Here,  too,  cost  factors  are  frequently  constructed,  such 
as  cost  per  military  man.  This  factor  is  found  from  an  aggregate  budgetary 
expenditure  for  pay  and  allowances,  and  an  aggregate  manpower  figure. 

If  applied  to  quasi-homogeneous  categories  of  manpower  or  materials, 
simple  cost  factors  provide  satisfactory  estimates  for  most  comparative  pur¬ 
poses.  It  should  be  emphasized  that  this  is  a  procedure  solely  for  estimating 
costs  of  given,  specified  methods  of  employing  these  resources.  It  says  nothing 
about  the  efficient  ways  of  combining  manpower  and  equipments  in  defense  ac¬ 
tivities.  This  is  a  different  and  more  penetrating  question.  If  analysis  is  to 
provide  findings  on  this  question,  it  must  attempt  experiments  or  simulations 
that  actually  vary  amounts  and  qualities  of  manpower  in  combination  with  mate¬ 
rials  and  equipments . 


Early  Estimates  for  Advanced  Systems 

Defense  cost  analysis  is  now  old  enough  to  have  a  histroy .  A  review  of  the 
record  of  approximately  the  first  decade  of  cost  estimating,  a  period  correspond¬ 
ing  closely  to  the  nineteen-fifties,  shows  a  variable  performance.  There  is  a 
bias  toward  underestimation  (reference  (i)).  The  degree  of  underestimation,  and 
the  variance  of  estimation  errors,  both  seem  to  be  related  to  the  earliness  of  the 
estimate  and  the  type  of  proposal.  Differences  between  early  estimates  and 
ultimately  realized  costs  have  been  small  for  systems  where  attempted  advances 
were  modest.  These  include,  for  example,  non-combatant  ships  and  cargo  air¬ 
craft.  Errors  in  early  estimates  are  ?vstematically  larger  for  new  combat  air¬ 
craft  with  advanced  radars,  communications,  and  weapons;  and  for  advanced 
rocket  propulsion  and  guidance  systems . 

Aside  from  the  partiality  and  advocacy  which  to  some  degree  affect  all  fore¬ 
casters,  what  accounts  for  this  record?  TNvo  major  areas  of  uncertainty  are 
distinguished.  One  is  uncertainty  concerning  the  time  and  difficulty  to  develop 
for  procurement  a  design  and  configuration  that  ultimately  achieves  a  promised 
performance.  The  other  is  pure  estimating  uncertainty  concerning  the  costs  of 
delivering  any  specified  physical  configuration.  There  is  a  good  prospect  of 
improving  this  development-and-cost  estimating  record,  for  proposals  where 
attempted  technological  advances  are  great,  by  focusing  early  estimation  on  in¬ 
formation  that  relates  cost  to  advances  in  performance  instead  of  costing  some¬ 
one's  early  design  specifications. 


Why  do  we  want  early  cost  estimates  for  proposals  "promising"  great 
advances?  At  this  stage  the  problem  is  far  different  from  procurement  selec¬ 
tion  where  both  contractor  and  proposal  must  be  selected  for  large  production 
runs.  By  contrast,  research  and  development  is  a  process  of  securing  infor¬ 
mation  about  the  real  capability  of  proposers  to  deliver  performance,  -  how, 
when,  and  at  what  cost.  It  is  a  process  that  buys  this  information  incrementally, 
year  by  year.  It  would  be  foolish  early  in  pre -development  to  act  as  if  we  can 
then  estimate  each  proposal's  cost  reliably  enough  to  make  a  single,  terminal 
selection  without  component  and  subsystem  development  yielding  test  data. 

The  reason  for  wanting  very  early  estimates  is  that  development  resources 
also  are  constrained.  Even  if  we  intend  to  buy  partial  or  sequential  develop¬ 
ment  on  similar  proposals  we  must  select  these  from  a  range  of  aspirants.  The 
cost  estimate  is  one  aid  in  preliminary  evaluation  of  development  proposals. 

As  estimates  are  revised  when  confronted  with  experience  they  become  credible 
for  force -structure  analyses  leading  to  production  decisions. 


Sensitivity 

All  studies  include  doubtful  features  that  cannot  be  satisfactorily  specified 
or  resolved  quantitatively  within  the  study  (reference  (j)).  The  character  of  the 
threat,  the  detection  range  of  a  sensor,  the  effectiveness  of  an  aircraft  against 
a  target,  the  cost  of  the  needed  support  activities  may  all  be  examples.  Each 
can  be  varied  in  magnitude  to  see  how  the  final  quantitative  results  are  influenced 
by,  or  are  "sensitive"  to,  possible  variability  in  the  doubtful  factor. 

This  exercise  can  be  very  revealing.  In  a  complicated  analysis  it  can 
greatly  reduce  the  obscurity  between  assumptions  and  findings  by  showing 
separately  the  features  to  which  results  are  and  are  not  responsive.  We  can 
thereby  indicate  where  the  more  costly  "mistakes"  may  be,  even  if  we  cannot 
estimate  a  strict  statistical  reliability.  Uncertainties  bearing  heavily  on  results 
become  options  for  further  research  and  experiment.  This  must  be  the  path  of 
progress  in  systems  analvses.  Most  formal  studies  are  more  synthesis  than 
analysis.  Each  synthesizes  much  existing  knowledge,  adds  a  bit  to  the  fund,  or 
helps  to  identify  efforts  that  will. 


SOME  AVENUES  FOR  FURTHER  WORK 

The  broad  and  pervasive  area  called  "support"  presents  difficult  problems 
in  measuring  resource  claims.  Apart  from  the  question  of  management  efficiency, 
and  how  it  may  be  improved,  it  is  difficult  simply  to  identify  the  incremental 
(dec remental)  physical  quantities  of  many  resource  services  associated  with  in¬ 
cremental  (decremental)  achievements  in  system  effectiveness.  When  we  look 
behind  the  warfare  system  "identifiers",  i.e..  the  hardware  items,  one  finds 
substantial  investment  and  operating  costs  for  "support"  activities  required  to 
sustain  their  capability.  Specific  and  necessary  services  are  partially  obscured 


in  the  supporting  military  structure.  Although  the  current  Department  of  Defense 
programming  system  has  made  a  contribution  to  unravelling  this  structure,  the 
format  does  not  automatically  answer  questions  a  systems  analysis  must  raise 
concerning  which  portions  of  support  activities  are  incrementally  linked  to  pos¬ 
sible  changes  in  a  system  or  force  structure. 

Another  difficulty  in  specifying  relevant  costs  occurs  when  a  system  directed 
to  one  objective  would,  if  selected,  increase  effectiveness  or  reduce  costs  of 
another  mission.  In  economic  analysis  these  are  external  economies  or  spill¬ 
overs.  Spillovers  within  the  market-and-price  system  consist  of  items  of  bene¬ 
fit  or  cost  that  are  experienced  by  others  as  a  result  of  one  decision-maker's 
choices,  which  he  does  not  take  into  account,  and  for  which  others  are  not 
charged  or  recompensed.  Although  in  different  form,  essentially  the  same 
phenomena  are  found  in  government,  including  the  defense  sector.  Spillovers 
to  other  activities  are  likely  to  be  larger  the  narrower  the  scope  of  alternative 
systems  that  enter  a  comparative  analysis  and,  thus,  the  larger  the  range  of 
other  activities  that  employ  similar  resources  or  provide  related  effectiveness . 
Hence  it  is  often  desirable  to  attempt  to  broaden  a  study  to  include  more  types 
of  effectiveness  and  reduce  the  allocation  of  costs  among  narrow  missions . 

For  some  resources,  including  categories  of  manpower,  there  are  ceilings 
cm  prices.  These  and  other  resources  are  rationed  or  assigned  among  alter¬ 
native  users  in  defense  on  some  basis  other  than  price-expressed  valuations. 
Assigned  users  do  not  have  to  pay  as  much  as  would  be  required  to  secure  them 
from  other  abortive  claimants  through  open  bidding.  If  prices  are  fixed  or  non¬ 
existent,  neither  the  external  acquisition  nor  internal  allocation  processes  yield 
good  information  on  relative  productivities  in  different  uses  and,  thus,  on  true 
alternative  cost.  In  this  environment,  further  exploration  of  technological  pos¬ 
sibilities  for  substitution  among  manpower  and  equipments  in  defense  programs 
will  very  likely  reduce  costs  or  improve  c  ipabilities.  There  is  a  growing  aware¬ 
ness  of  possible  payoffs  in  efforts  to  analyze  manpower  utilization  and  to  improve 
the  quantification  of  system  claims  on  manpower. 
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